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ABSTRACT 

In response to core indicators of effectiveness 
developed by the American Association of Community Colleges (AACC) , 
the Colorado Community College and Occupational Education System 
(CCCOES) initiated a system-wide project to develop core indicators 
specific to the state's colleges. The project was intended to reflect 
the immediat e needs for the public and the legislature to understand 
the performance of the CCCOES in many key areas. Based on the AACC 
indicators and meet ings with coll ege pres i dents , vice presidents , 
deans, and research directors, the CCCOES developed the following 
core indicators: (1) student goal attainment; (2) a global measure of 

persistence, completion, and transfer; (3) specific persistence 
rates; (A) degree and certificate completion; (5) transfer; (6) a 
transfer rate for each college; (7) graduate employment in the 
workforce; (8) success in subsequent, related coursework for basic 
skills students; (9) participation rate in service area for credit 
programs; (10) employer assessment of students; (11) demonstration of 
critical 1 i teracy skills; (12) demons t rat ion of citizenship skills; 
(13) client assessment of programs and services; and (lA) 
responsiveness to community needs . The CCOES was given the 
respons ibi lity for developing the first nine indicators, while the 
final five are the responsibility of individual colleges. Systemwide 
data on outcomes as of fall 199A for the first nine indicators are 
included. (TGI) 
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Background 

The landscape of higher education, especially across community colleges, is 
changing dramatically. Federal legislation, including the Student Right to Know 
regulations which mandate reporting of graduatiotx statistics, dramatic changes in 
institutional eligibility requirements for federal financial aid, evolving requirements for 
accreditation, mandates for Perkins vocational funds reporting, changing student 
populations, and increased scrutiny by state governments and the public have created a 
challenging environment for community colleges. Some institutions have responded to 
this agenda by identifying performance indicators and building effectiveness models. 
Experience shows some models are uncomplicated, while some are complex. Other 
institutions, simply ignore effectiveness indicators until an impending accreditation 
review or required state or federal report reawakens interest. 

Simultaneously, stakeholders in our colleges are continually faced with making 
judgements about institutional effectiveness. Students, community leaders, legislators, 
business and industry officials, faculty, and administrators make choices, about our 
institutions daily. As they make these choices, they intersperse information from their 
experience about high quality (effective) performance and poor quality (ineffective) 
performance into the effectiveness equation. These individuals may draw their 
judgements from direct experience with a college or through information gleaned from a 
secondary source, i.e., newspapers, television advertising, college catalogs, 
conversations with friends, etc. Human nature virtually guarantees that when 
individuals lack direct information they will not delay passing judgement on a college. 
Instead, their judgments will come from secondary sources that usually have little or 
nothing to do with institutional quality. 

Community colleges typically turn to the factors that are easily quantified to 
respond to questions about institutional goals. Historically, these measures have 
included growth in programs, budgets, and enrollments. Enrollment growth is the most 
frequently used indicator of effectiveness, but its use as the sole indicator of 
effectiveness opens the community college to attack by critics who claim that quantity, 
as expressed by enrollment, camouflages a lack of quality. Other indicators, besides 
growth, may better serve the long-term health of the community colleges, if these 
indicators account for differences in size, location, funding, campus culture, and 
administrative philosophies. 

Colleges without a framework to address effectiveness also find it difficult to 
argue persuasively in the state and federal policy and budget setting arenas. This 
vacuum has spurred legislators and other policy-makers to create a sweeping, and 
often bewildering, array of accountability requirements. In Colorado, the state currently 
pursues accountability through HB 1187. The Colorado Commission on Higher 
Education (CCHE) now serves as mediator between colleges and the legislature. On 
the federal level, the impact of requirements found under recent federal legislation will 
require each campus to adopt a framework for educating internal and external 
audiences about institutional effectiveness. 
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Project Planning and Methodology 

In response to these issues, Colorado Community College and Occupational 
Education System (CCCOES) staff, with the Presidents' Program and Planning 
Committee, initiated a system-wide project designed to develop core indicators of 
effectiveness, hoping that this project would provide a much-needed focus on 
effectiveness through identifying a small array of indicators. A framework chosen for 
this project is a report prepared by the American Association of Community Colleges 
(AACC) in 1994, Community Colleges; Core Indicators of Effectiveness. This report 
was the result of work by community college executive officers, university professors, 
and higher education officials. CCCOES staff held a series of meetings and 
discussions throughout 1995 with college presidents, vice presidents and deans, and 
research and assessment directors from all system colleges, examining AACC's 13 core 
indicators and carefully re-drawing operational definitions of these indicators for the 
system. 

The consensus from the parties consulted in developing the project is that 
AACC's 13 core indicators reflect the immediate needs for the public and the legislature 
to understand the performance of our colleges in many key areas. Meanwhile, these 
indicators, if adopted nationally, are also characterized by generalizability across 
institutions, ease and efficiency of use, relevance to community college mission, and 
significance to multiple customers. 

While developing the operational definitions of the core indicators, CCCOES 
staff examined all the available data sources to make sure that the indicators could be 
carried out regularly across the system and over the years to come. Because of this 
scrutiny, reliable and valid data sources were chosen to develop this report, and some 
indicators were left for individual colleges to develop. It is anticipated that each 
CCCOES campus will proceed with development of non-system generated indicators. 

The data bases used for this report include; (1) CCCOES Student Information 
System (SIS); (2) CCHE Student Unit Record Data System (SURDS) extract files; (3) 
CCCOES VE135 Follow-Up system; (4) Integrated Postsecondary Data System 
(IPEDS) reports for CCCOES institutions; and (5) U.S. Census data, 1990. 

Commercial software (SPSS PC+, FOCUS for Open VMS, and Quattro Pro ) was used 
to extract data, conduct statistical analyses, and to develop graphs. 
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Presentation of CCCOES Indicators 

The order and number in which indicators are presented here differ from previous 
presentation schemes. The new order corresponds to the availability across the system 
and to the strategy of combining several indicators to present a more comprehensive 
picture of student success. Indicator 2, for example, combines data on persistence, 
degree completion, and transfer rates to provide a global overview of system 
effectiveness. Insertion of this new indicator expands the total number to 14. indicators 
and locus of responsibility in developing these indicators are presented below. This 
report contains systemwide data for indicators 1 through 9. it is anticipated that 
campuses will be able to supplement this report with data which address Indicators 10 
through 14. 
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Core Indicator 1 : Student Goal Attainment 



Measure; 



Data Source (s): 
Technical Notes: 



Results; 



The proportion of Fall 1994 students (full-time and part-time) 
expressing intentions in the areas below. 

• Associate degrees 

• Vocational certificate 

• T ransf er 

• Skill Upgrade 

• Personal enrichment 

• Unknown 

SURDS extract file; and SIS 

The total number of students found in this indicator should match 
Fall 1994 headcount enrollment as reported by colleges to CCHE. 
Student demographics were also extracted from the colleges' 
official IPEDS files. Student intention, primary degree programs, 
students enrolled in fall 1994, and their course loads were 
extracted from SIS. In 1994-95 CCCOES colleges collaboratively 
implemented a new application form designed to better trap student 
goals at entry, especially transfer intentions and employment 
intentions. For this report, the actual program a student was 
enrolled in Fall 1994 is listed to supplement incomplete information 
student intention. It is anticipated that a more complete view of 
student intent will be available after system colleges have 
implemented the new application form. 

. Overall, one-fifth (20.2%) of the students said that their 
intention for attending a Colorado community college 
attendance was job-related. 

• One quarter of the students expressed intention to transfer 
to four-year institutions, and 20 percent of them enrolled in 
Associate of Arts (AA) or Associate of Science (AS) 
programs. 

• Nearly 1 8 percent of the students expressed intentions in 
personal interest or enrichment, and over one-third of them 
simply did not know or did not want to identify their goals. 

• Nearly two-thirds of the students attempted less than 1 2 
credit hours. The remainder attended full-time. 
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Student Degree Preferences 
CCCOES, Fall 1994 




Gender 

CCCOES, Fall 1994 
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Core Indicator 2: Persistence, Completion, and Transfer 

Measure: The data for this indicator are taken from a CCHE report on 

graduation, transfer, and persistence. The cohorts used were 
those who enrolled in a community college as first-time, full-time, 
and degree-seeking students. 



Data Source(s): CCHE Graduation/Transfer/Persistence Report, April 3, 1995. 



Technical Notes: Indicators 3, 4, and 5 are presented together to show the overall 

educational attainment of CCCOES students after the 4th fall of 
original enrollment. Overlap between completion, transfer, and 
persistence is presented graphically for the 1990 cohort in the 
graph below. Additionally, aggregate data for five entering cohorts 
(1986 to 1990) are also graphed below. 

Results; The overall rate of persistence, completion, and transfer for the 

1990 cohort is 40 percent. About 17 percent of the cohort 
completed a 2-year degree or certificate and a total of 20 percent 
transferred to Colorado public 4-year institutions. Among them, 4 
percent of the students both completed a degree or certificate and 
transferred. Nearly 8 percent of the cohort were still enrolled in a 
community college after the 4th fall of original enrollment. 
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Completers, Transfers, and Persisters 
1990 Entering Cohort 
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Core Indicator 3: 
Measure: 

Data Source(s): 
Technical Notes: 

Results: 



Persistence 

The proportion of Fall 1986, 1987, 1988, 1989, and 1990 first-time, 
full- time, degree or certificate seeking students who were still 
enrolled at their original institution three falls later. 

CCHE Graduation/Transfer/Persistence Report, April 3, 1995 



The data for this indicator are taken from an Ad-hoc CCHE report 
on graduation, transfer, and persistence. The cohorts used were 
those who enrolled in a community college as first-time, full-time, 
and degree- seeking students. It is necessary to examine 
persistence rates as reported in this indicator together with 
indicators 4 and 5 (completion and transfer) to get a complete 
picture about CCCOES student attainment. 

S/stemwide persistence rates after the 4th fall of original 
enrollment increased by 36 percent over a five-year period. 



Persistence 
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Core Indicator 4: Degree and Certificate Completion 

Measure; The proportion of Fall 1 986, 1 987, 1 988, 1 989, and 1 990 first-time, 

full- time, degree or certificate seeking students who by Spring 
1994 earned Associate degrees or/and vocational certificates. 



Data Source(s); CCHE Graduationn'ransfer/Persistence Report, April 3, 1995. 

Technical Notes: Data for this indicator are taken from the same CCHE report on 

graduation, transfer, and persistence. The cohorts used were those 
who enrolled in a community college as first-time, full-time, and 
degree- seeking students. The statistic includes those who 
completed and then transferred to a four-year institution and those 
who completed and did not transfer. Again, it is necessary to 
examine completion rates as reported in this indicator with 
indicators 3 and 5 (persistence and transfer) to get a complete 
picture about student attainment. 



Results: System-wide degree and certificate completion rates have been as 

high as 22 percent, with individual colleges' rates approaching 40 
percent for certain years. Completion figures vary from institution 
to institution. 



Degree or Certificate Completion 
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Core Indicator 5: Transfer Rate 

Measure: 



The proportion of entering Fall 1986, 1987, 1988, 1989, and 1990 
first- time, full-time, degree or certificate seeking students who 
transferred to Colorado public four-year institutions. 

Data Source(s): CCHE Graduationmansfer/Persistence Report, April 3, 1995. 



Technical Notes: 



Results: 



The data for this indicator are taken from the same CCHE report on 
graduation, transfer, and persistence. The cohorts used were 
those who enrolled in a community college as first-time, full-time, 
degree-seeking students. The number of completions includes 
those who completed and then transferred to a four-year institution 
and those who transferred without completing ^o-year degree or 
certificate. It is necessary to examine completion rates as reported 
in this indicator with indicators 3 and 4 (persistence and degree and 
certificate completion) to get a complete picture about student 
attainment. 

The system-wide transfer rates range from 16 to 20 percent, with 
some institutions having transfer rates over 20 percent and others 
producing single-digit rates. 
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Core Indicator 6: Performance at Transfer 
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Measure: A comparison of cumulative GPA's at the community college by 

1993-94 degree and certificate recipients transferring to Colorado 
public 4-year institutions with their GPA’s earned at the 4-year 
transfer institutions during the first year after transfer. 

Data Source(s): CCCOES Research and Planning from CCHE SURDS extract file. 

Technical Notes: This graph displays the average cumulative GPA's of transfer 

students at the community college and at Colorado 4-year public 
institutions. "Total" is an unduplicated count, but individuals may be 
duplicated across degree type. That is, an individual who received 
both a certificate and an AAS degree in the same year is 
represented in both groups but included only once in the total. The 
N's represent the number of individuals who actually transferred. In 
some cases, due to a small amount of missing data (less than 
15%), the GPA'S are based on N's slightly smaller than those 
reported. 

Results: The largest drop in GPA (3.1 to 2.6) is for the group of community 

college certificate recipients. 



; FI rst|(^ar«^|^ 


' Degree^pe"->'^'- 


%ef6te| 




Certificate 


3.1 


2.6 


AAS 


3.2 


3.0 


AS/AA 


3.3 


3.0 


AGS 


3.2 


3.0 





3EST COPY AVAILABLf 



13 



Post-Transfer, Short-term GPA’s 

1 993-94 COCOES Graduates 
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Core Indicator 7: Graduate Employment in the Work Force 

Measure; The proportion of 1993-94 Associate in Applied Science degree and 

vocational certificate graduates in the categories below. 



• employed related 

• employed unrelated 

• unemployed seeking 

• unemployed not seeking and 

• continuing education. 



Data Source(s); COCOES VE1 35 follow-up data reported by each college. 

Technical Notes: The data for this indicator are from the CCCOES-maintained data 

base (VE135). Community college AAS and vocational certificate 
recipients are surveyed by each college to determine their 
employment and education status. Two-thirds of the 1993-1994 
graduates (66.6%) were contacted in this follow-up survey. The 
percentages of employment and continuing education are 
calculated separately. Therefore, some individuals could be 
employed and continuing their educations simultaneously. 



Results: 




A total of 96 percent of the contacted graduates v5rho indicated they 
are available for work were employed. Among them, 82 percent 
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are employed in jobs related to their training in community colleges. 
Nearly one quarter (23.8%) of the graduates contacted reported 
that they are continuing education. 



Placement of AAS and Certificate Grads 
CCCOES, 1993-1994 




Proportion of Continuing Education 
AAS and Certificate Graduates, 1993-94 



(23.79%) 



(76.21%) 




Status six months after the 1993-94 academic year 
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Subsequent College-Level Success Rates 
Fall 1993 BSE Students 



BSE Mathematics 
College Mathematics 
BSE English 
College English 




Core Indicator 8: Success in Subsequent, Related Coursework for Basic Skills 
Students 

Measure: The proportion of Fall 1993 basic skills education English and rnath 

completers who subsequently enrolled in any college-level English 
or math classes and completed these courses with a grade of C or 
better. 



Data Sourc€T(s); CCCOES Research and Planning from SIS. 

Technical Notes: Students' initial enrollment in basic skills English and math courses 

in fall 1993 was captured to form two groups. The grades they 
received from the basic skills courses were recorded. In the 
subsequent term (Spring 1994) those who completed basic skills 
with £‘s or better and continued in college level English and math 
respectively were tracked again to see whether they completed the 
college level courses successfully. 



Results: Among those taking basic skills courses in fall 1993, 70 percent of 

the English students and nearly two-thirds (64%) of the math 
students completed the courses with a grade of Q. or better. In the 
subsequent term (Spring 1994), the successful completers of basic 
skills courses in the previous term were followed in their college 
level English and math courses respectively. Seventy-six (76) 
percent of the continuing English group and 72 percent of the 
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continuing math group could complete the college level courses 
successfully with a grade of Q or better. 



Core Indicator 9: Participation Rate in Service Area for Credit Programs 

Measure: The proportion of the total population aged 18 and over residing in 

the college service area who register for at least one credit hour In 
the college credit programs during the 1993-94 academic year. 

Data Source(s): CCCOES Research and Planning from SIS; 1990 U.S. Census 

data. 



Credit Participation Rates for Adults 
in College Service Areas, 1993-94 



Participants (4.47%) 




(95.53%) 
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Technical Notes; The SIS system was used to calculate the total unduplicated 

headcount enrollment in college credit programs for academic year 
1993-94, along with the total credit hours attempted. College 
service areas were defined according to the State Board policy, and 
the total population aged 18 and over was calculated for service 
areas using the 1990 U.S. census data. The participation rate is a 
rough estimate based on unduplicated headcount and service area 
population. 

Results: The estimated figure shows that nearly 5 percent of the population 

aged 18 and over were served by Colorado system community 
colleges in credit programs. This figure does not account for those 
adults enrolled in non-credit programs and other community-based 
activities. 

Core indicator 1 0: Employer Assessment of Students 

Measure: An overall satisfaction rating of CCCOES graduates personal 

attributes and job skills by employers and/or supervisors 

Data Source(s): CCCOES-deyeloped accountability surveys. 

Technical Notes: Colleges conduct surveys on employers/supervisors of their 

graduates every year to satisfy the existing accountability reporting 
requirements. No systemwide data are available for this indicator. 

Core Indicator 11: Demonstration of Critical Literacy Skilis 

Core Indicator 12: Demonstration of Citizenship Skills 

Core Indicator 13: Client Assessment of Programs and Services 

Core Indicator 14: Responsiveness to Community Needs 

Measure: No single statistic satisfies the intent of these indicators, neither are 

systemwide data available. Campuses will develop— or have 
developed-data for these indicators for inclusion in their own 
reports. 
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